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Four  recent  capstone  projects  by  students  in  the  U.S.  Naval  Academy’s  (USNAJ  Department  of  Computer  Science  offer 
some  interesting  insights  into  methodologies  for  information  assurance  (LA).  This  article  looks  at  the  tasks  and  challenges  of 
each  project  and  consolidates  the  experiences  into  lessons  learned  for  designing  and  implementing  software  or  systems  that  incor¬ 
porate  the  LA  concepts  of  confidentiality,  data  integrity,  authentication,  and  system  availabilily  [1], 


One  USNA  requirement  (for  comput¬ 
er  science  or  IT  undergraduates)  is  a 
capstone  project.  Students — in  groups  of 
three  or  four  on  a  project  of  their  choos¬ 
ing — must  find  a  customer,  define  require¬ 
ments,  and  meet  key  milestone  dates  in 
providing  a  software  or  system  artifact. 
Projects  require  about  150  hours  per  per¬ 
son  and  must  be  completed  and  fully  doc¬ 
umented  within  the  15 -week  semester. 

Over  tire  past  two  years,  there  has  been 
increased  student  motivation  to  choose  IA- 
related  projects.  Like  software  or  systems 
engineering  projects  in  other  fields,  stu¬ 
dents  found  it  especially  challenging  to 
define  customer  requirements  and  meet 
expectations  and  milestones.  Faculty  use 
these  challenges  as  learning  opportunities 
by  allowing  students  to  make  their  own 
project  decisions,  even  if  poor  decision¬ 
making  leads  to  a  mid-project  failure, 
because  these  failures  will  teach  the  stu¬ 
dents  much  more  than  a  perfectly  executed 
plan.  Students  found  drat  taking  on  pro¬ 
jects  in  die  IA  field  of  study  created  addi¬ 
tional  challenges  in  subject  matter  knowl¬ 
edge,  system  design,  and  implementation. 

Team  I :  Training  Database 
with  Multi-Level  Views 

The  first  project  required  students  to 
develop  a  training  and  personnel  database 
drat  provides  proper  authentication  at  an 
undetermined  number  of  organizational 
or  data  visibility  levels  and  minimizes  rep¬ 
etition  of  data  entry  by  using  data  nor¬ 
malization.  This  group  found  the  require- 
ments-gathering  process  to  be  fairly 
straightforward,  as  the  customer  under¬ 
stood  the  concept  of  a  database  with  mul¬ 
tiple  levels  of  security.  These  requirements 
included  allowing  designated  individuals 
to  input  and  view  multiple  training  cours¬ 
es  as  well  as  providing  status  reports  to 
higher-level  managers. 

This  team  designed  tire  software  with 
an  initial  authentication  scheme  and  then 
created  a  secure  session  drat  verified  cre¬ 
dentials  before  displaying  data.  Read,  write, 
and  modify  rules  were  given  based  on  data¬ 


base  views,  combining  multiple  tables  in 
various  views.  The  database  administrator 
programmed  these  views,  which  had  the 
capability  of  providing  granular  permis¬ 
sions.  Originally,  the  team  planned  to 
hard-code  permissions  as  read  and  modify 
for  all  levels  higher  than  tire  supervisor, 
meeting  the  customer  requirements. 
However,  after  initial  design  review,  they 
realized  that  these  requirements  may  later 
change;  therefore,  the  team  redesigned 
access  control  by  giving  the  database 
administrator  the  ability  to  set  visibility  by 
level  or  by  allowing  the  overriding  of  per¬ 
missions.  This  additional  flexibility  added 
tire  capability  to  produce  a  report  by  giv¬ 
ing  full  permissions  by  person,  group,  and 
supervisory  levels,  as  well  as  highlighting 
all  overridden  permissions. 

Team  2:  Emergency 
Notification  System 

The  next  project1  required  an  undeter¬ 
mined  number  of  individuals  and  organi¬ 
zations  to  receive  emergency  notification 
of  events  based  on  input  from  city,  state, 
nationwide,  or  worldwide  sensors.  Some 
events  only  need  to  be  seen  by  the  local 
emergency  services  while  others  required 
visibility  for  governors,  the  military,  or 
federal  officials.  Some  events  are  simply 
logged,  while  higher-priority  events  may 
require  confirmation  from  the  appropriate 
source  and  the  ability  to  forward  data  to 
higher  levels  for  additional  action. 

This  team  also  had  a  challenge  with  the 
design  and  implementation  of  permission 
and  authentication  methods.  Unlike  the 
first  group,  however,  these  students  did 
not  initially  incorporate  authentication  at 
die  design  phase.  While  they  understood 
die  requirements  to  have  multiple  levels  of 
reporting  based  on  the  users’  position  and 
authentication,  the  team  decided  to  put 
diis  off  until  the  implementation  phase. 
Because  specifying  the  design  is  arguably 
die  most  difficult  step  in  the  software 
engineering  process,  many  students  simply 
want  to  get  started  with  the  implementa¬ 
tion  phase.  These  students  started  imple¬ 


menting  code  based  on  a  poorly  elaborat¬ 
ed  design.  They  split  up  work,  each  build¬ 
ing  separate  Web  pages  for  a  type  of 
emergency  reporting  required.  This  result¬ 
ed  in  disparate  pages  that  looked  and 
operated  differently  and  had  no  means  of 
accepting  dynamic  changes.  In  addition, 
the  team  realized  that  they  designed 
authentication  by  page  or  view  rather  than 
providing  a  consolidated,  centralized 
means  of  authentication  and  a  data  per¬ 
mission  schema. 

Though  reluctant  to  scrap  six  weeks  of 
work,  the  team  ultimately  chose  to  begin 
from  scratch  and  start  again  at  the  require¬ 
ments  phase.  At  this  point,  they  developed 
formal  interview  questions  for  the  cus¬ 
tomer  and  wrote  a  concise  statement  that 
encompassed  all  requirements.  They  then 
took  each  sentence  or  phrase  and  turned  it 
into  a  well-defined  requirement  placed  in  a 
requirements  implementation  and  testing 
matrix.  From  this  matrix,  the  students  cre¬ 
ated  a  design  that  incorporated  every 
requirement.  These  requirements  speci¬ 
fied  authentication  and  visibility  schemas 
for  each  view.  After  further  analysis,  the 
team  was  able  to  design  a  method  for  cen¬ 
tralized  authentication  and  visibility  with  a 
small  change  to  tire  database  schema. 

The  team  was  able  to  re -accomplish 
requirements  analysis  and  design  in  only  a 
week,  and  was  able  to  implement  the 
backend  database  in  another  week.  The 
team  admitted  drat  they  had  their  doubts 
about  whether  they  could  finish  die  pro¬ 
ject  on  time,  but  were  surprised  to  see  the 
ease  of  implementing  and  testing  well- 
defined  requirements  and  design. 

Team  3:  Cybersecurity 
Competition  Framework 

This  project — stemmed  from  a  Polytech¬ 
nic  Institute  of  New  York  University 
competition — had  students  from  numer¬ 
ous  schools  downloading  various  cyberse¬ 
curity  and  digital  forensics  exercises  that 
were  timed  and  graded  for  accuracy  and 
completeness. 

USNA  students  felt  that  drey  could 
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Game-Changing  Tools  and  Practices 


improve  the  competition  by  designing  a 
better  interface  for  serving  and  grading 
die  cyber  challenges.  As  such,  they  set  out 
to  create  a  Web-based  software  framework 
that  served  various  scenarios  and  received 
team  responses  for  an  IA  competition. 
Requirements  included  authentication  and 
proper  visibility  of  scenarios  for  an  unde¬ 
termined  number  of  teams,  competition 
referees,  scenarios,  and  team  responses. 
Additionally,  since  they  were  creating  a 
framework  for  a  hacking  competition, 
they  had  to  design  a  system  that  main¬ 
tained  the  integrity  and  availability  of  the 
data  despite  possible  hijack  attempts  from 
less  scrupulous  teams. 

In  consultation  with  the  faculty,  stu¬ 
dents  chose  to  both  build  the  framework 
diat  was  to  serve  the  cyber  challenges  and 
create  individual  challenges  as  a  proof  of 
concept  for  their  serving  framework.  As 
such,  they  assigned  two  team  members  to 
build  the  framework,  while  two  others 
independently  built  challenges.  The  re¬ 
quirements  called  for  a  broad  range  of 
possible  cyber  challenges  including:  digital 
forensics  of  disk  images,  analysis  of  net¬ 
work  traffic,  analysis  of  software  code  vul¬ 
nerabilities,  and  the  identification  and  mit¬ 
igation  of  operating  system  and  applica¬ 
tions  security  configurations. 

Like  the  others,  this  team  started  by 
gaining  detailed  requirements  for  which 
protection  against  common  software  vul¬ 
nerabilities  was  key.  They  quickly  realized 
diat  system  security  had  to  be  built  into 
die  design  process.  Widi  diis  additional 
requirement,  they  realized  the  design 
would  be  the  most  difficult  aspect  of  their 
project  and  allocated  additional  time  for 
diis  milestone.  Before  designing  in  securi¬ 
ty,  the  team — both  to  protect  their  infra¬ 
structure  and  to  develop  challenges  for 
the  competition  framework — had  to 
understand  how  hackers  use  vulnerabili¬ 
ties  to  get  into  systems.  Each  student 
chose  to  specialize  in  specific  network, 
operating  system,  application,  and  data¬ 
base  security  techniques.  To  better  under¬ 
stand  these  techniques,  students  reviewed 
previous  coursework,  examined  DoD  and 
National  Security  Agency  (NS A)  security 
guides2,  and  interviewed  network  security 
administrators.  They  found  that  the  most 
difficult  portion  of  securing  an  applica¬ 
tion  against  hackers  is  not  the  actual 
implementation  of  a  specific  configura¬ 
tion  or  fix,  but  in  thinking  like  die  hacker 
and  predicting  how  people  will  use  poten¬ 
tial  vulnerabilities  to  disrupt  operations. 

Though  the  team  found  implementa¬ 
tion  of  the  database  and  associated  views 
to  be  relatively  trivial,  they  found  die  doc¬ 
umentation  process  to  be  challenging. 


Documentation  had  to  include  reasons  for 
dieir  design  decisions  and  security  set¬ 
tings,  so  that  future  maintainers  could  add 
features  but  still  capitalize  on  the  security 
features  built  into  die  framework. 

Students  noted  that  there  was  great 
value  in  understanding  and  implementing 
die  security  guides.  While  their  IA  course 
had  many  hands-on  experiences,  it  was 
only  through  the  course  of  this  project 
diat  they  realized  the  complexity  of  secur¬ 
ing  applications. 

Team  4:  Cyber  Defense 
Exercise 

In  this  project,  students  were  required  to 
design  and  implement  a  complete  network 
based  on  an  intricate  set  of  constraints. 
After  implementation,  students  had  to 
operate  and  defend  this  network  against 
NS  A  experts  posing  as  attackers.  Called 
die  Cyber  Defense  Exercise  [2],  the  com¬ 
petition  is  modified  annually  to  increase 
die  cybersecurity  skills  required  of  student 
participants.  Several  years  ago,  the  focus 
was  on  active  defense,  while  the  more 
recent  exercises  have  focused  on  die  trade¬ 
offs  that  need  to  be  made  between  limited 
resources,  operations  of  a  network,  securi¬ 
ty,  time,  and  expertise  required. 

Students  were  provided  with  a  40-page 
directive  spelling  out  die  rules  of  die  com¬ 
petition  along  widi  listing  die  network  ser¬ 
vices  diat  must  be  provided  during  a  week- 
long  exercise  (and  die  points  to  be  deduct¬ 
ed  if  diese  services  were  either  not  opera¬ 
tional  or  had  security  compromises). 
Essentially,  students  were  given  a  very 
detailed  requirements  document  with  total 
freedom  to  produce  any  design.  Though 
students  were  asked  to  turn  in  their  designs, 
referees  only  verified  diat  diey  met  bud¬ 
getary  constraints.  Cross-referencing  re¬ 
quirements  to  design  was  a  task  left  totally 
to  each  competing  student  group. 

Though  students  were  not  required  to 
gather  requirements  from  an  external  cus¬ 
tomer,  diey  did  have  to  interpret  die  direc¬ 
tive  and  design  a  complete  network  given 
die  assumptions,  constraints,  and  require¬ 
ments.  Students  were  challenged  with  cre¬ 
ating  a  design  that  could  provide  users 
with  a  number  of  services,  such  as  e-mail, 
chat,  Web,  databases,  file  servers,  and  mis¬ 
sion-specific  applications.  This  design  had 
to  remain  operational  while  withstanding 
attacks  from  NSA  network  experts  posing 
as  hackers. 

Student  team  members  had  taken  bodi 
networking  and  IA  courses,  yet  there  was 
still  a  great  deal  of  knowledge  needed  for 
die  secure  design  of  an  operational  net¬ 
work.  Students  augmented  their  knowledge 


of  secure  design  widi  NSA  security  guides, 
Defense  Information  Systems  Agency 
security  checklists3,  and  various  security- 
specific  books  and  references.  Despite 
these  numerous  references,  students  were 
still  challenged  with  consolidating  this 
information  and  meeting  die  required  con¬ 
straints.  Perhaps  die  students’  greatest  chal¬ 
lenge  was  verifying  die  security  of  dieir 
design  and  implementation,  which  was 
done  creating  a  test  environment  that  mim¬ 
icked  die  actions  of  die  attackers.  The  team 
used  security  testing  tools  like  the 
Metasploit  Framework  (which  provides 
pre-packaged  exploits)  to  test  if  the  system 
is  vulnerable  to  attack.  Students  also  used 
other  sites  like  <www.milwOrm.com>  to 
test  dieir  system  against  additional  poten¬ 
tial  exploits;  however,  the  use  of  these 
more  advanced  techniques  required  great 
experience  and  training. 

In  addition  to  the  testing  of  security, 
students  had  challenges  in  ensuring  diat  the 
tightened  security  did  not  impact  network 
operations.  The  students  found  this  to  be  a 
great  challenge.  This  balance  between  con¬ 
tinued  operations  versus  increased  security 
involves  business  case  and  risk  analysis,  a 
skill  diat  generally  requires  expertise  in 
both  network  security  and  the  mission  area 
supported  by  die  system. 

Lessons  Learned 

Through  these  student  projects,  we  can 
learn  a  number  of  security-related  lessons 
about  gaining  requirements,  as  well  as 
designing  and  implementing  IA-focused 
systems  or  software. 

Design  Authentication  and  a  Data 
Permissions  Schema  Early 

Nearly  all  application  or  system  develop¬ 
ment  requires  authentication  methods. 
Students  found  that  the  best  results  were 
achieved  by  planning  for  both  position- 
level  and  personal-level  authentication  for 
data  visibility  during  the  design  phase. 
Even  when  requirements  only  call  for  sim¬ 
ple  authentication,  customers  tend  to  ask 
for  a  layered  authentication  by  data  type, 
organizational  position,  or  data  view. 
Rework  tends  to  be  extensive  and  time- 
consuming.  Planning  for  authentication  in 
the  design  phase  of  systems  will  likely  save 
time  and  resources  in  the  long  run. 

Use  Security  Guides 

Students  found  that  despite  more  than 
100  classroom  hours  spent  learning  about 
networks  and  IA  techniques,  additional 
application-specific  information  is  re¬ 
quired  when  designing  and  developing  IA- 
focused  applications  or  systems.  The  NSA 
and  the  Defense  Information  Systems 
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Software  Defense  Application 

Through  die  lessons  learned  by  USNA  students,  this  article  is  a  refresher  for  defense 
software  developers  on  why  it  is  important  to  design  authentication  and  data  permis¬ 
sions  early,  follow  security  guides,  use  existing  techniques  for  security  testing,  do 
regression  testing  as  changes  are  made,  ensure  drat  customers  understand  security 
costs  and  tradeoffs,  recognize  the  unintended  impact  of  users  on  security,  and  thor¬ 
oughly  document  all  elements  of  security  architecture. 


Agency  produce  security  guides  for  vari¬ 
ous  operating  systems  and  applications. 
These  guides  have  been  produced  and 
tested  by  numerous  experts  and  can  com¬ 
plement  the  developers’  knowledge  to 
meet  design  specifications  for  applications 
requiring  a  cybersecurity  focus. 

Test  Applications  Against  Known 
Security  Frameworks 

Relatively  few  software  or  system  develop¬ 
ers  have  the  skills  required  to  test  system 
designs  and  implementations  against  well- 
known  attacks  using  exploits  in  systems 
availability,  data  confidentiality,  and 
integrity.  Rather  than  recreating  exploits 
drat  may  require  a  greater  effort  than  actu¬ 
ally  securing  the  system  or  application, 
system  testers  are  encouraged  to  use  exist¬ 
ing  security  testing  frameworks.  While 
tliese  testing  frameworks  can  demonstrate 
potential  holes  in  security,  they  should  be 
used  in  concert  with  secure  programming 
techniques,  design,  as  well  as  documented 
and  tested  security  techniques. 

Plan  for  Regression  Testing 

Students  working  on  tliese  projects  noted 
die  need  for  an  updated,  descriptive  test 
plan  and  follow-on  regression  software 
testing.  This  is  true  in  any  software  appli¬ 
cation,  but  tends  to  be  highlighted  in  secu¬ 
rity-focused  applications.  Security  en¬ 
hancements  are  rarely  made  in  a  single 
place.  Instead,  these  changes  are  made  in 
die  operating  system,  database  framework, 
application,  and  various  configuration 
files.  Students  found  that  a  single  change 
in  any  one  of  diese  areas  forced  regres¬ 
sion  errors  that  were  very  difficult  to 
detect  without  a  well-formulated  and 
implemented  test  plan.  Students  learned 
that  this  testing  needs  to  be  done  as 
changes  are  made,  or  it  becomes  necessary 
to  back  out  entire  blocks  of  changes  to 
find  the  root  cause  of  bugs. 

Manage  Security  Expectations 

Customers  generally  understand  those 
requirements  and  expected  outcomes  that 
are  directly  related  to  their  subject  matter 
expertise.  Through  diese  projects,  stu¬ 
dents  noted  that  customers  expect  an 
application  to  be  secure,  but  do  not  under¬ 
stand  die  resource  costs  or  operations 
tradeoffs  required  to  make  this  a  reality. 
Students  noted  die  need  to  manage  cus¬ 
tomer  security  expectations  in  the  require¬ 
ments  phase  and  later  in  the  design  phase. 
Students  believed  that  the  best  way  to 
manage  security  requirements  and  associ¬ 
ated  customer  expectations  was  to  provide 
a  security,  operations,  and  resource  matrix 
that  cross-references  security  trade-offs. 


Understand  Security  Impact  on 
Operations 

Even  after  managing  customers’  security 
expectations  and  implementing  security 
(expected  to  exceed  requirements),  stu¬ 
dents  found  that  users  can  have  die  great¬ 
est  unintended  impact  on  security.  In  test¬ 
ing  these  applications  with  actual  users  (as 
tliey  would  use  them),  students  found  that 
users  will  bypass  security,  in  turn  impairing 
operations  or  user-expected  procedures. 
These  user-caused  workarounds  can 
reduce  security  effectiveness  and  give  the 
application  owner  a  false  sense  of  securi¬ 
ty.  Students  learned  that  for  security  con¬ 
trols  to  remain  effective,  designers  must 
understand  existing  user  processes  and 
procedures — and  then  design  security 
architectures  around  these  or  build  them 
in  a  user-friendly  alternative. 

Document  Reasons  for  Security 
Architecture 

like  many  software  professionals,  students 
found  project  documentation  among  die 
most  challenging  processes.  As  a  learning 
tool,  students  were  required  to  make 
changes  to  projects  based  on  documenta¬ 
tion  that  eidier  they  created  or  (in  some 
cases)  was  created  by  other  student  teams. 
Though  effective  documentation  is  always 
challenging,  students  found  diis  difficulty 
was  magnified  when  trying  to  modify  secu¬ 
rity  architectures.  Ultimately,  they  noted 
diat  understanding  die  security  architecture 


documentation  is  not  enough  to  effectively 
make  changes  to  security  widiout  impact¬ 
ing  operations  or  functionality.  Instead,  stu¬ 
dents  found  it  easier  to  effectively  manage 
security  changes  when  they  had  documen¬ 
tation  that  also  explained  die  reason  for 
decisions,  limitations  in  technology,  the 
state  of  the  intended  operating  system’s 
security,  and  operational  or  process  trade¬ 
offs  associated  with  security  decisions.^ 
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Notes 

1.  This  project  was  inspired  by  die  2007 
film  “Live  Free  or  Die  Hard,”  where 
terrorists  took  control  of  emergency 
services,  die  electric  grid,  and  city-wide 
traffic  signals. 

2.  For  access  to  these  guides,  visit: 
<www.nsa.gov/ia/guidance/ security 
_configuration_guides>. 

3.  See  <http://iase.disa.mil/stigs/check 
list>. 


About  the  Authors 


Lt.  Col.  Thomas  A. 
Augustine,  D.Sc.,  is 

retired  from  the  United 
States  Air  Force  after  a 
career  in  communica¬ 
tions  and  information. 
His  most  recent  assign¬ 
ment  was  as  an  assistant  professor  of 
computer  science  at  the  USNA,  special¬ 
izing  in  networks  and  IA. 

E-mail:  thomas.augustine@ 
hotmail.com 


Lori  L.  DeLooze,  Ph.D., 

retired  from  the  United 
States  Navy  as  a  career 
information  professional. 
She  is  currently  an  assis¬ 
tant  professor  of  com¬ 
puter  science  at  the 
USNA,  specializing  in  software  engineer¬ 
ing  and  IA. 

572  Holloway  RD,  Stop  9F 
Annapolis,  MD  21402 
Phone:  (4 10)  293-6820 
E-mail:  delooze@usna.edu 


September/October  2010 


www.stsc.hill.af.mil  I  5 


